feat(cala): Phase 6 — archive + vitals UI#140
Merged
Conversation
Upgrade the archive worker from a latest-value Map to per-name tiered Float32Array rings (design §9.1): L1 full-resolution recent + L2 block-averaged older. Adds `request-timeseries` / `timeseries` protocol variants so the dashboard can pull either tier. Existing `archive-dump` shape (events + latest-value metrics) is unchanged, keeping task 24's dashboard working without edits. Capacities and stride are all overridable via `workerConfig`, no magic numbers. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `NeuronEventIndex` — bounded drop-oldest per-neuron ring of every structural event the neuron participates in (design §9.2), so "show me the history of neuron 47" is O(index) instead of a scan of the global event ring. Wires a third bus subscriber in the archive worker plus a `request-events-for-neuron` / `events-for-neuron` protocol pair. Also bumps the archive worker's local-bus `maxSubscribers` default to 8 so the upcoming footprint-history subscriber fits without another edit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the archive-side receiver for the hybrid log-spaced + change-triggered footprint scheme (design §9.3). The new `FootprintHistoryStore` keeps per-neuron drop-oldest rings of `(t, sparse A column)` snapshots; typed-array payloads are retained by reference to avoid per-snapshot copies. Harvests footprints from three sources through the archive worker's local event bus: (1) birth events, (2) merge-survivor events, (3) split children, plus a new `footprint-snapshot` `PipelineEvent` variant that W2 will emit on the log-spaced schedule in task 5. Adds a `request-footprint-history` / `footprint-history` protocol pair for the scrubber UI in Phase 7. Default sizing matches the §9.3 ~5 MB per-session budget. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fit worker now emits the five header-bar vitals (cell_count, fps, memory_bytes, residual_l2, extend_queue_depth) every vitalsStride frames (default 8). residual_l2 is computed from `fitter.step()`'s return; fps is wall-clock-derived over the last interval so it reflects what the user sees on the sparklines. Metric names live in a new `lib/vitals.ts` module shared with the upcoming vitals bar so emitter and UI can't drift. Also adds `calaMemoryBytes()` to the cala-core wasm-adapter — a single source of truth for the WASM heap size now that a consumer (W2) actually needs it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the log-spaced floor of the §9.3 hybrid scheme: a new `FootprintSnapshotScheduler` emits `footprint-snapshot` events at ages 1, 2, 4, 8, … frames after each neuron's birth, using the latest footprint attached to a mutation (register / merge / split) as its cached snap. The change-triggered branch (‖A_curr − A_last_snap‖_F drift) is left as a deliberate TODO — it needs a new wasm-bindgen accessor on `Fitter` to read per-component A columns. Until that lands the log-spaced schedule gets us scrubber data during quiet periods and exercises the archive's footprint-history store end-to-end. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the main-thread archive client with three new promise-based request methods that forward to the W4 queries added in tasks 1-3: - requestTimeseries(name) → tiered sparkline data for a metric - requestEventsForNeuron(id) → per-neuron structural history - requestFootprintHistory(id) → per-neuron (t, sparse A) scrubber data Refactors the existing dump path onto a generic `issueRequest` helper so every reply kind shares one requestId-correlation store without type drift. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the §12 header vitals bar: five Canvas-based sparkline widgets bound to a new `vitals-store` that polls the archive client every 500 ms for cell_count / fps / memory_bytes / residual_l2 / extend_queue_depth. L1 + L2 tiers merge into a single flat window (default 120 samples ≈ 60 s of history) so the component doesn't know about the retention scheme. Lifecycle is tied to the run: the bar spins up an archive client on transition to running, polls while the run is active, and tears down on stop/error. Sparklines auto-scale per series so each vital gets its full vertical range regardless of magnitude. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the §12 event feed: a structured, grid-laid-out log of every pipeline event from the dashboard store — kind, id, time, and a compact human-readable description per row. Birth / deprecate / reject rows get distinct accent colors so the feed reads as a narrative rather than a wall of text. Format helpers are extracted into `event-format.ts` and unit-tested directly so the visual component stays untouched by string-formatting churn. Task 9 composes this + VitalsBar + SingleFrameViewer into the final dashboard layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Composes VitalsBar + SingleFrameViewer + EventFeed into a single grid layout (design §12): vitals along the top, preview canvas on the left, event feed on the right, each cell independently scrollable. App.tsx hands off to `DashboardLayout` on any active-run state; the old SingleFrameViewer side-panel + inline event list come out since their roles are now covered by the dedicated components. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extracts the Phase 3 cold-start E2E's inline `run_extend_cycle` helper into a public crate function at `extending::driver::run_cycle`, then wraps it behind a new `Extender` wasm-bindgen class. The browser W3 worker can now own a `ResidualRingBuf`, push residuals per fit frame, and call `runCycle(fitter, queue)` on whatever cadence it chooses — proposals land on the shared `MutationQueueHandle` ready for `drainApply`. Exports `Extender` through `@calab/cala-core`. The Phase 3 test switches to the shared driver to prove numerical parity: same code path serves both the native cold-start recovery proof and the browser worker. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fit worker now owns an `Extender` + residual ring: on every frame it pushes the residual, and on `extendCycleStride` (default 32 frames ≈ 1 s at 30 fps) it calls `Extender.runCycle(fitter, queue)`. Real `register` proposals land on the Rust mutation queue and get applied on the next `drainApply`, advancing the fitter's epoch and populating `cell_count` / `numComponents` — the vitals bar actually moves now. The per-cycle proposal count is published as `extend.proposed` so the event feed shows extend activity even before structural events surface. W3's heartbeat stub stays in place for the orchestrator lifecycle handshake. Architectural trade-off: running the cycle inside W2 avoids a cross-worker snapshot transport (design §7.2). True W2/W3 separation needs a WASM binding that serializes the Snapshot across workers + a `Fitter.drainApplyEvents()` to surface `register` payloads as `birth` events — both deferred to Phase 7. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `e2e/phase6-extend.e2e.test.ts`, mirroring the Phase 5 exit harness (real AVI bytes, real SabRingChannel, real worker modules) but configured with a non-zero `extendCycleStride` so fit's new extend path actually fires. Asserts: * Extender.runCycle is driven every stride * fitter.drainApply is called per cycle → epoch advances * `extend.proposed` metric events reach W4 with the expected count Also fixes a bug the E2E surfaced: task 11's `runExtendCycleIfDue` pushed proposals onto the Rust mutation queue but never triggered `drainApply` — the existing drain loop only fires when the JS-side queue has items. Now the extend path drain-applies inline so epoch advances as soon as the cycle proposes anything. Phase 5 E2E mock picks up a no-op StubExtender since the fit worker's Extender import became unconditional; no behavior change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a main-thread authoring path for pipeline mutations. New
types and flow:
* `UserMutation` in the runtime's worker-protocol (narrow to
`deprecate` for Phase 6 — register / merge need a footprint
picker, deferred).
* `RuntimeController.pushUserMutation(m)` forwards to the fit
worker as a `user-mutation` inbound.
* Fit worker handles it by pushing through the Rust-side queue
(`pushDeprecate`), drain-applying so epoch advances, and
publishing a `deprecate` structural event so the UI feed shows
the user's action.
UI affordance: a "Deprecate latest" button in the event feed
toolbar that targets the most recently born neuron id. The real
click-a-footprint UX lands with the Phase 7 overlay.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops `public/coi-serviceworker.js` — an inlined, BSD-licensed service worker that re-issues top-level navigations with COOP + COEP headers so the Pages deploy boots `crossOriginIsolated` and `SharedArrayBuffer` works without server-side header control. Registered synchronously in `index.html` before the module script so the page is isolated before any SAB-using worker is constructed. Vite dev + preview already set the same headers directly, so this is a no-op there. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sk 16)
Ships `e2e/phase6-exit.e2e.test.ts`: drives the full decode → fit
→ archive pipeline on a real miniscope AVI and asserts every
Phase-6-shipped surface is reachable end-to-end:
- vitals timeseries populate (tiered store from task 1)
- per-neuron event index resolves a birth (task 2)
- footprint history returns entries (task 3)
- extend.proposed metrics + structural events land in the archive
dump (tasks 4 + 5 + 11)
- user-authored deprecate mutations reach the fit worker (task 13)
- no worker errors during or after the run
Also fixes `pixel_size_um` missing from W2's metadataJson (task 11
introduced an Extender construction that parses RecordingMetadata
but run-control.ts only forwarded height/width). Surfaced during
live browser testing; without the fix the fit worker panics on
init in production.
Updates `.planning/CALA_DESIGN.md` §12 with the Phase 6 exit date,
the shipped artifact list, and explicitly deferred items (real
birth events need `Fitter.drainApplyEvents`, cross-worker
snapshot transport, ε change-trigger, Playwright, click-a-
footprint UX).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
W1's `frame-processed` heartbeat hardcodes `epoch: 0n` (it has no view of the fit pipeline's epoch), but `run-control` was routing that to `recordFrameProcessed` — so the viewer label read `epoch 0` forever while the fit pipeline silently advanced. Move the dashboard listener onto the fit worker, which is the only worker that knows the real epoch. W1's listener stays for `frame-preview` only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `initCalaCore` resolver now reads `mod.memory` to expose `calaMemoryBytes()` (Phase 6 task 23 / vitals bar), and the adapter re-exports the new `Extender` binding (task 23). The mocked init resolver needed to return a stub `WebAssembly.Memory` instead of `undefined`, and the stub module needed to export an `Extender` class. Adds one test for the new `calaMemoryBytes()` helper. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 6 (design §12) ships the "feels alive" layer on top of the Phase 5 streaming pipeline:
Exit proven by `apps/cala/e2e/phase6-exit.e2e.test.ts` — full end-to-end run asserting tiered timeseries, per-neuron event queries, footprint history, archive dump, and user mutation round-trip.
Deferred to Phase 7+ (documented in design §12): real `birth`/`merge` `PipelineEvent`s (needs `Fitter.drainApplyEvents` binding), cross-worker snapshot transport so extend truly runs in W3, ε change-triggered footprint snapshots, Playwright harness, click-a-footprint mutation UX.
Test plan
🤖 Generated with Claude Code